{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MLflow \n",
    "\n",
    "The sktime custom model flavor enables logging of sktime models in MLflow format via the `sktime.utils.mlflow_sktime.save_model()` and `sktime.utils.mlflow_sktime.log_model()` methods. These methods also add the `pyfunc` flavor to the MLflow Models that they produce, allowing the model to be interpreted as generic Python functions for inference via `sktime.utils.mlflow_sktime.pyfunc.load_model()`. This loaded PyFunc model can only be scored with a DataFrame input. You can also use the `sktime.utils.mlflow_sktime.load_model()` method to load MLflow Models with the sktime model flavor in native sktime formats.\n",
    "\n",
    "The `pyfunc` flavor of the model supports sktime predict methods `predict`,  `predict_interval`, `predict_proba`, `predict_quantiles`, `predict_var`.\n",
    "\n",
    "The interface for utilizing a sktime model loaded as a `pyfunc` type for generating forecasts requires passing an exogenous regressor as Pandas DataFrame to the `pyfunc.predict()` method (an empty DataFrame must be passed if no exogenous regressor is used). The configuration of predict methods and parameter values passed to the predict methods is defined by a dictionary to be saved as an attribute of the fitted sktime model instance. If no prediction configuration is defined `pyfunc.predict()` will return output from sktime `predict` method. Note that for `pyfunc` flavor the forecasting horizon `fh` must be passed to the fit method.\n",
    "\n",
    "Predict methods and parameter values for `pyfunc` flavor can be defined in two ways: \n",
    "- `Dict[str, dict]` if parameter values are passed to `pyfunc.predict()`, for example  `{\"predict_method\": {\"predict\": {}, \"predict_interval\": {\"coverage\": [0.1, 0.9]}}`\n",
    "- `Dict[str, list]`, with default parameters in predict method, for example  `{\"predict_method\": [\"predict\", \"predict_interval\"}` (Note: when including `predict_proba` method the former approach must be followed as `quantiles` parameter has to be provided by the user) \n",
    "- If no prediction config is defined `pyfunc.predict()` will return output from sktime `predict()` method\n",
    "\n",
    "Signature logging for sktime from a non-pyfunc artifact will not function correctly for `predict_interval` or `predict_quantiles`. The output of the native sktime model flavor for these methods is not a recognized signature type due to the MultiIndex column structure of the returned DataFrame. MLflow's ``infer_schema`` will function correctly if using the ``pyfunc`` flavor of the model, though."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Setup\n",
    "### 1.1 Config"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_path = \"model\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import mlflow\n",
    "\n",
    "from sktime.datasets import load_longley\n",
    "from sktime.forecasting.naive import NaiveForecaster\n",
    "from sktime.split import temporal_train_test_split\n",
    "from sktime.utils import mlflow_sktime"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 Load sample data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "y, X = load_longley()\n",
    "y_train, y_test, X_train, X_test = temporal_train_test_split(y, X)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Example usage of native `sktime flavor` and `pyfunc flavor`\n",
    "\n",
    "### 2.1 Create prediction config for pyfunc flavor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "coverage = [0.8, 0.9]\n",
    "quantiles = [0.1, 0.9]\n",
    "\n",
    "pyfunc_predict_conf = {\n",
    "    \"predict_method\": {\n",
    "        \"predict\": {},\n",
    "        \"predict_interval\": {\"coverage\": coverage},\n",
    "        \"predict_proba\": {\"quantiles\": quantiles},\n",
    "        \"predict_quantiles\": {},\n",
    "        \"predict_var\": {},\n",
    "    }\n",
    "}"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Train and save model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/benjamin/anaconda3/envs/sktime-dev/lib/python3.8/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.\n",
      "  warnings.warn(\"Setuptools is replacing distutils.\")\n"
     ]
    }
   ],
   "source": [
    "with mlflow.start_run():\n",
    "    forecaster = NaiveForecaster()\n",
    "    forecaster.fit(\n",
    "        y_train,\n",
    "        X=X_train,\n",
    "        fh=[1, 2, 3],\n",
    "    )\n",
    "    forecaster.pyfunc_predict_conf = pyfunc_predict_conf\n",
    "\n",
    "    mlflow_sktime.save_model(forecaster, model_path)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 Load model\n",
    "\n",
    "#### 2.3.1 Native sktime flavor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "loaded_model = mlflow_sktime.load_model(model_path)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.3.2 Pyfunc flavor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "loaded_pyfunc = mlflow_sktime.pyfunc.load_model(model_path)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.4 Generate predictions\n",
    "\n",
    "#### 2.4.1 Native sktime flavor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1959    66513.0\n",
       "1960    66513.0\n",
       "1961    66513.0\n",
       "Freq: A-DEC, dtype: float64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loaded_model.predict(X=X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"4\" halign=\"left\">Coverage</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"2\" halign=\"left\">0.8</th>\n",
       "      <th colspan=\"2\" halign=\"left\">0.9</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>lower</th>\n",
       "      <th>upper</th>\n",
       "      <th>lower</th>\n",
       "      <th>upper</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1959</th>\n",
       "      <td>64719.913711</td>\n",
       "      <td>68306.086289</td>\n",
       "      <td>64211.598663</td>\n",
       "      <td>68814.401337</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1960</th>\n",
       "      <td>63977.193051</td>\n",
       "      <td>69048.806949</td>\n",
       "      <td>63258.327017</td>\n",
       "      <td>69767.672983</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1961</th>\n",
       "      <td>63407.283445</td>\n",
       "      <td>69618.716555</td>\n",
       "      <td>62526.855956</td>\n",
       "      <td>70499.144044</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          Coverage                                          \n",
       "               0.8                         0.9              \n",
       "             lower         upper         lower         upper\n",
       "1959  64719.913711  68306.086289  64211.598663  68814.401337\n",
       "1960  63977.193051  69048.806949  63258.327017  69767.672983\n",
       "1961  63407.283445  69618.716555  62526.855956  70499.144044"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loaded_model.predict_interval(X=X_test, coverage=coverage)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-12-19 10:07:18.984171: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA\n",
      "To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
      "2022-12-19 10:07:19.137912: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n",
      "2022-12-19 10:07:19.137942: I tensorflow/compiler/xla/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n",
      "2022-12-19 10:07:19.961244: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory\n",
      "2022-12-19 10:07:19.961430: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory\n",
      "2022-12-19 10:07:19.961446: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.\n",
      "2022-12-19 10:07:21.052618: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2022-12-19 10:07:21.053031: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n",
      "2022-12-19 10:07:21.053101: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcublas.so.11'; dlerror: libcublas.so.11: cannot open shared object file: No such file or directory\n",
      "2022-12-19 10:07:21.053161: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcublasLt.so.11'; dlerror: libcublasLt.so.11: cannot open shared object file: No such file or directory\n",
      "2022-12-19 10:07:21.054924: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcusolver.so.11'; dlerror: libcusolver.so.11: cannot open shared object file: No such file or directory\n",
      "2022-12-19 10:07:21.054986: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcusparse.so.11'; dlerror: libcusparse.so.11: cannot open shared object file: No such file or directory\n",
      "2022-12-19 10:07:21.055042: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudnn.so.8'; dlerror: libcudnn.so.8: cannot open shared object file: No such file or directory\n",
      "2022-12-19 10:07:21.055054: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1934] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.\n",
      "Skipping registering GPU devices...\n",
      "2022-12-19 10:07:21.055334: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA\n",
      "To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Quantiles_0.1</th>\n",
       "      <th>Quantiles_0.9</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1959</th>\n",
       "      <td>64719.914062</td>\n",
       "      <td>68306.085938</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1960</th>\n",
       "      <td>63977.191406</td>\n",
       "      <td>69048.804688</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1961</th>\n",
       "      <td>63407.281250</td>\n",
       "      <td>69618.718750</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Quantiles_0.1  Quantiles_0.9\n",
       "1959   64719.914062   68306.085938\n",
       "1960   63977.191406   69048.804688\n",
       "1961   63407.281250   69618.718750"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred_dist = loaded_model.predict_proba(X=X)\n",
    "y_pred_dist_quantiles = y_pred_dist.quantile(quantiles)\n",
    "y_pred_dist_quantiles"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"2\" halign=\"left\">Quantiles</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>0.05</th>\n",
       "      <th>0.95</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1959</th>\n",
       "      <td>64211.598663</td>\n",
       "      <td>68814.401337</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1960</th>\n",
       "      <td>63258.327017</td>\n",
       "      <td>69767.672983</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1961</th>\n",
       "      <td>62526.855956</td>\n",
       "      <td>70499.144044</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Quantiles              \n",
       "              0.05          0.95\n",
       "1959  64211.598663  68814.401337\n",
       "1960  63258.327017  69767.672983\n",
       "1961  62526.855956  70499.144044"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loaded_model.predict_quantiles(X=X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1959</th>\n",
       "      <td>1.957628e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1960</th>\n",
       "      <td>3.915256e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1961</th>\n",
       "      <td>5.872885e+06</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 0\n",
       "1959  1.957628e+06\n",
       "1960  3.915256e+06\n",
       "1961  5.872885e+06"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loaded_model.predict_var(X=X_test)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.4.2 Pyfunc flavor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>predict__0</th>\n",
       "      <th>predict_interval__Coverage__0.8__lower</th>\n",
       "      <th>predict_interval__Coverage__0.8__upper</th>\n",
       "      <th>predict_interval__Coverage__0.9__lower</th>\n",
       "      <th>predict_interval__Coverage__0.9__upper</th>\n",
       "      <th>predict_proba__Quantiles_0.1</th>\n",
       "      <th>predict_proba__Quantiles_0.9</th>\n",
       "      <th>predict_quantiles__Quantiles__0.05</th>\n",
       "      <th>predict_quantiles__Quantiles__0.95</th>\n",
       "      <th>predict_var__0</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1959</th>\n",
       "      <td>66513.0</td>\n",
       "      <td>64719.913711</td>\n",
       "      <td>68306.086289</td>\n",
       "      <td>64211.598663</td>\n",
       "      <td>68814.401337</td>\n",
       "      <td>64719.914062</td>\n",
       "      <td>68306.085938</td>\n",
       "      <td>64211.598663</td>\n",
       "      <td>68814.401337</td>\n",
       "      <td>1.957628e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1960</th>\n",
       "      <td>66513.0</td>\n",
       "      <td>63977.193051</td>\n",
       "      <td>69048.806949</td>\n",
       "      <td>63258.327017</td>\n",
       "      <td>69767.672983</td>\n",
       "      <td>63977.191406</td>\n",
       "      <td>69048.804688</td>\n",
       "      <td>63258.327017</td>\n",
       "      <td>69767.672983</td>\n",
       "      <td>3.915256e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1961</th>\n",
       "      <td>66513.0</td>\n",
       "      <td>63407.283445</td>\n",
       "      <td>69618.716555</td>\n",
       "      <td>62526.855956</td>\n",
       "      <td>70499.144044</td>\n",
       "      <td>63407.281250</td>\n",
       "      <td>69618.718750</td>\n",
       "      <td>62526.855956</td>\n",
       "      <td>70499.144044</td>\n",
       "      <td>5.872885e+06</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      predict__0  predict_interval__Coverage__0.8__lower  \\\n",
       "1959     66513.0                            64719.913711   \n",
       "1960     66513.0                            63977.193051   \n",
       "1961     66513.0                            63407.283445   \n",
       "\n",
       "      predict_interval__Coverage__0.8__upper  \\\n",
       "1959                            68306.086289   \n",
       "1960                            69048.806949   \n",
       "1961                            69618.716555   \n",
       "\n",
       "      predict_interval__Coverage__0.9__lower  \\\n",
       "1959                            64211.598663   \n",
       "1960                            63258.327017   \n",
       "1961                            62526.855956   \n",
       "\n",
       "      predict_interval__Coverage__0.9__upper  predict_proba__Quantiles_0.1  \\\n",
       "1959                            68814.401337                  64719.914062   \n",
       "1960                            69767.672983                  63977.191406   \n",
       "1961                            70499.144044                  63407.281250   \n",
       "\n",
       "      predict_proba__Quantiles_0.9  predict_quantiles__Quantiles__0.05  \\\n",
       "1959                  68306.085938                        64211.598663   \n",
       "1960                  69048.804688                        63258.327017   \n",
       "1961                  69618.718750                        62526.855956   \n",
       "\n",
       "      predict_quantiles__Quantiles__0.95  predict_var__0  \n",
       "1959                        68814.401337    1.957628e+06  \n",
       "1960                        69767.672983    3.915256e+06  \n",
       "1961                        70499.144044    5.872885e+06  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loaded_pyfunc.predict(X_test)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Model deployment example\n",
    "\n",
    "### 3.1 Create experiment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022/12/19 10:07:21 INFO mlflow.tracking.fluent: Experiment with name 'Test Sktime' does not exist. Creating a new experiment.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MLflow run id: ec94d157fe354c1bbdd4dec0898e1ed6\n"
     ]
    }
   ],
   "source": [
    "artifact_path = \"model\"\n",
    "\n",
    "mlflow.set_experiment(\"Test Sktime\")\n",
    "\n",
    "with mlflow.start_run() as run:\n",
    "    forecaster = NaiveForecaster()\n",
    "    forecaster.fit(y_train, X=X_train, fh=[1, 2, 3])\n",
    "    forecaster.pyfunc_predict_conf = pyfunc_predict_conf\n",
    "\n",
    "    mlflow_sktime.log_model(sktime_model=forecaster, artifact_path=artifact_path)\n",
    "\n",
    "run_id = run.info.run_id\n",
    "print(f\"MLflow run id: {run_id}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 Deploy pyfunc model to local REST API endpoint\n",
    "- Open a terminal window and cd into `examples`directory\n",
    "- In the terminal run: `mlflow models serve -m runs:/<RUN_ID>/model --env-manager local --host <HOST>`\n",
    "    - where you substitute `<RUN_ID>` by the `run_id` and `<HOST>` by the network address to listen on (e.g. `127.0.0.1`) \n",
    "- More details here: https://www.mlflow.org/docs/latest/cli.html#mlflow-models-serve"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 Request predictions from local REST API endpoint\n",
    "\n",
    "- For more details see: https://www.mlflow.org/docs/latest/models.html#built-in-deployment-tools\n",
    "\n",
    "#### 3.3.1 JSON input using `dataframe_split` field with pandas DataFrame in the `split` orientation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'dataframe_split': {'index': [0, 1, 2, 3], 'columns': ['GNPDEFL', 'GNP', 'UNEMP', 'ARMED', 'POP'], 'data': [[112.6, 482704.0, 3813.0, 2552.0, 123366.0], [114.2, 502601.0, 3931.0, 2514.0, 125368.0], [115.7, 518173.0, 4806.0, 2572.0, 127852.0], [116.9, 554894.0, 4007.0, 2827.0, 130081.0]]}}\n"
     ]
    }
   ],
   "source": [
    "host = \"127.0.0.1\"\n",
    "url = f\"http://{host}:5000/invocations\"\n",
    "\n",
    "X_test = X_test.reset_index(drop=True)\n",
    "json_data = {\"dataframe_split\": X_test.to_dict(orient=\"split\")}\n",
    "print(json_data)\n",
    "\n",
    "# # Comment in the below lines to run the prediction request\n",
    "# import requests\n",
    "# response = requests.post(url, json=json_data)\n",
    "# response.json()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.3.2 JSON input using `dataframe_records` field with pandas DataFrame in the `records` orientation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'dataframe_records': [{'GNPDEFL': 112.6, 'GNP': 482704.0, 'UNEMP': 3813.0, 'ARMED': 2552.0, 'POP': 123366.0}, {'GNPDEFL': 114.2, 'GNP': 502601.0, 'UNEMP': 3931.0, 'ARMED': 2514.0, 'POP': 125368.0}, {'GNPDEFL': 115.7, 'GNP': 518173.0, 'UNEMP': 4806.0, 'ARMED': 2572.0, 'POP': 127852.0}, {'GNPDEFL': 116.9, 'GNP': 554894.0, 'UNEMP': 4007.0, 'ARMED': 2827.0, 'POP': 130081.0}]}\n"
     ]
    }
   ],
   "source": [
    "json_data = {\"dataframe_records\": X_test.to_dict(orient=\"records\")}\n",
    "print(json_data)\n",
    "\n",
    "# # Comment in the below lines to run the prediction request\n",
    "# response = requests.post(url, json=json_data)\n",
    "# response.json()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.3.3 CSV input using valid `pd.DataFrame` csv representation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ",GNPDEFL,GNP,UNEMP,ARMED,POP\n",
      "0,112.6,482704.0,3813.0,2552.0,123366.0\n",
      "1,114.2,502601.0,3931.0,2514.0,125368.0\n",
      "2,115.7,518173.0,4806.0,2572.0,127852.0\n",
      "3,116.9,554894.0,4007.0,2827.0,130081.0\n",
      "\n"
     ]
    }
   ],
   "source": [
    "headers = {\n",
    "    \"Content-Type\": \"text/csv\",\n",
    "}\n",
    "data = X_test.to_csv()\n",
    "print(data)\n",
    "\n",
    "# # Comment in the below lines to run the prediction request\n",
    "# response = requests.post(url, headers=headers, data=data)\n",
    "# response.json()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "sktime-skbase-310",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  },
  "vscode": {
   "interpreter": {
    "hash": "1ab6896984b6aa500b8009633c692bca601cfe3e50e0ab79a8a59539ceef9c7a"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
